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PATENT 

ATTORNEY DOCKET NO: 07072/919001 

MULTIPLE STORAGE ARRAY CONTROL 
Background of the Invention 
5 The invention relates generally to the control of 

multiple storage devices , such as disk arrays. 

Storage systems have grown enormously in both size 
and sophistication in recent years. These systems can 
typically include many large disk drive units controlled by 
10 a complex, multi-tasking, disk drive controller such as the 
EMC Symmetrix disk drive controller. A large scale disk 
drive system can typically receive commands, such as I/O 
requests, from a number of host computers and can control a 
number of disk drive mass storage devices, each mass storage 
15 device capable of storing in excess of several gigabits of 
data . 

There is every reason to expect that both the 
sophistication and the size of the disk drive systems will 
continue to increase. As the systems increase in 

20 complexity, so does a user's reliance upon the system for 
fast and reliable recovery and storage of data. 
Accordingly, the user typically uses data throughput and 
speed of response as a primary criteria for evaluating 
performance of the disk drive systems. As a result, mass 

25 storage devices and the controllers which drive them have 
become quite sophisticated in trying to improve command 
response time. Systems such as the EMC Symmetrix disk drive 
controller system thus incorporate a large cache memory, and 
other techniques to improve the system throughput . 

3 0 Of course, with the increase in size and complexity 

of such disk drive systems, comes a tremendous increase in 
the number of command requests making monitoring of the 
performance of the systems more difficult. 



Summary of the Invention 
The invention relates to a network architecture 
which facilitates the monitoring and control of a number of 
storage systems (e.g., arrays) by one or more clients. 

In a general aspect of the invention, the network 
architecture includes at least one storage management 
server, connected between the storage systems and a number 
of clients, for providing operation status information 
associated with the storage systems to at least one of the 
clients. 

Among other advantages of the storage management 
server, is the ability for a client to access (e.g., via an 
Intranet) the storage network and check the status of the 
storage systems for any problems which might affect 
throughput of data requested from a particular storage 
system. As the size of storage systems continue to 
increase, management responsibilities of the storage systems 
can become distributed. That is, different users of a 
system will have different responsibilities with respect to 
the objects stored within the system. In a storage system 
or storage array configured in this way, the storage 
management server facilitates the distribution of these 
responsibilities. In addition, a client can determine the 
configuration of the storage system, including the type, 
properties and status of storage connectivity devices (e.g., 
hubs, switches) and hosts, as well as, the version of 
software code or update used by the storage system. 
Moreover, distribution of the management of the storage 
systems is completely transparent to the clients. 

With networks having numerous host computers 
connected to a large storage system (e.g., greater than 1 
terabyte,) a tremendous number of input/output (I/O) 
requests are being made to and from the host computers at 



any given time. For example, when used with such networks, 
the storage management server allows a client to determine 
that a large number of I/O requests are being made through 
one channel of the network. The client can then consider 
5 the option of running an application using another storage 
system, thereby improving data throughput. 

In another aspect of the invention, two or more 
storage management servers may be required to manage and 
distribute the information to clients. Such applications 
10 include large storage applications having many storage 
devices and host computers. In such applications, each 
storage management server provides information relating to 
the operation status of the storage devices to the clients. 
Embodiments of these aspects of the invention may 
15 include one or more of the following features. 

The storage management server includes a poller for 
gathering the information relating to the operation status 
of the storage devices. Among other responsibilities, the 
poller polls each of the storage devices at predetermined 

2 0 intervals (as frequently as once per minute) to maintain the 

current status of the operation of each of the storage 
devices. Thus, status information for each storage device 
is guaranteed to be current or "fresh." 

The storage management server includes an object 
25 store, which serves as a central repository for storing the 
information relating to the operation status of the storage 
devices and an object server for distributing the 
information relating to the operation status of the storage 
devices to the clients. 

3 0 The storage management server further includes a 

security component for limiting access by a client to the 
information stored at the storage management server. In 
this way, access privileges to one or more of the storage 
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devices can be limited to a particular user or groups of 

users. The storage management server further also includes 

a web server for facilitating communication with the 

clients, via the Intranet. 
5 Each client further includes a graphical user 

interface for displaying the information relating to the 

operation status of the storage devices. 

The host computers and storage devices can be of 

different types. For example, one host computer could be a 
10 UNIX-based system, while another is based on Windows NT. 

Similarly, for example, the storage systems can be different 

versions of EMC's Symmetrix storage systems. 

Where multiple management storage servers are used, 

the network architecture may further include a naming 
15 service, connected to each of the storage management 

servers, to determine which of the central repositories of 

the storage management servers includes the information 

relating to the operation status of the storage devices of 

interest . 

2 0 Another aspect of the invention relates to a method 

of managing a storage system including storage devices, the 
storage system communicating data to and from host 
computers. The method includes the following steps. A 
storage management server is provided between a number of 
25 clients and the storage devices. The storage management 

server collects information relating to the configuration of 
the storage system and provides that information to at least 
one of the clients. 

Embodiments of this aspect of the invention may 

3 0 include one or more of the following features. 

Providing information relating to the operation 
status of the storage devices includes using a poller to 
gather the information relating to the operation status of 
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the storage device. The information relating to the 
operation status of the storage devices is stored in a 
central repository on the storage management server. An 
object server distributes the information relating to the 
5 operation status of the storage devices to the clients. 

A poller is used to poll each of the storage devices 
at predetermined intervals to maintain the current status of 
the operation of each of the storage devices. 

In certain embodiments/ information relating to the 
10 operation status of storage connectivity devices, which 

connect the hosts to the storage devices is also provided. 

Other features and advantages of the invention will 
become apparent from the following description, including 
the claims and the drawings. 

15 Brief Description of the Drawings 

Fig. 1 is a block diagram of a network architecture 

of the invention. 

Fig. 2 is a block diagram of the storage management 

server of the network architecture of Fig. 1. 
20 Fig. 3 is an example of a display screen of the 

graphical user interface for accessing the storage 

management server of Fig. 1. 

Fig. 4 is an alternative embodiment of a network 

architecture of the invention. 



2 5 Detailed Description 

Referring to Fig. 1, a network architecture 10 
includes an enterprise disk storage 12 connected to host 
computers (hereafter, hosts) 14a-14e through a 
communications network 13. The term "enterprise" as used 

3 0 here means that the disk storage is configured to allow 

multiple connectivity by, for example, hosts provided by 
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different vendors. As shown here, disk storage system 12 
includes individual storage arrays 12a- 12e, which may be, 
for example, one of a number of different Symmetrix systems, 
products of EMC Corporation, Hopkinton, MA. Each of the 
5 storage arrays 12a- 12e typically has an excess of 1 gigabit 
of memory and is logically divided, in accordance with known 
techniques, into a plurality of logical volumes. Each 
storage array 12a- 12e can thus have a plurality of logical 
volumes, for example, four, eight, or more logical volumes 

10 in a single physical disk drive element. Details concerning 
the architecture and operation of these systems are found, 
for example, in Symmetrix Product Manuals for Models 5500, 
52XX, 5100, 3500, 32XX, and 3100) all of which are 
incorporated herein by reference. 

15 The communications network may be, for example, a 

Fibre channel network allowing any of host computers 14a- 14c 
to communicate with any of storage arrays 12a-12e. Fibre 
channel network may be configured as a loop, a fabric having 
hubs or switches, or combinations of both. The hubs or 

2 0 switches represent storage connectivity devices for 

forwarding data packets to appropriate ports based on the 
address of the packet. 

Interconnecting the hosts 14a- 14e and the storage 
arrays 12a- 12e are disk drive controllers (not shown) , for 

25 example, that which is manufactured by EMC Corporation and 
known as the Symmetrix controller. 

Network architecture 10 also includes a storage 
management server 16 which is connected to and communicates 
with each of the hosts 14a- 14e via an appropriate agent 18a- 

30 18e stored on the host. As is known in the art, an agent is 
a program that performs information gathering or processing 
in the background. In particular, each agent 18a-18e has 
the well-defined task of gathering and providing information 



to management server 16 associated with the operation status 
of the storage array associated with the host running the 
agent. As will be described in greater detail below, 
management server 16 stores and makes accessible to a number 
5 of clients 20a-20e the operation status information of the 
storage arrays. Clients 2 0a-2 0e communicate with management 
server 16 through an on-line object request broker 22 (e.g., 
Voyager ORB, a product of ObjectSpace, Inc., Dallas, TX) . 
Object request broker 2 2 serves as a remote access adapter 

10 interface for ensuring that a client is able to access 
management server 16. Other object request brokers 22 
including RMI and CORBA are equally suitable for use for 
linking a client to management server 16. Thus, clients 
2 0a-2 0e can monitor the status of any or all storage arrays 

15 14a- 14e from virtually anywhere in the world, or at least 

where Intranet access is available. In particular, each of 
clients can independently access management server and 
determine the status of any or a number of storage systems 
12a- 12e. For example, a client 20a can determine whether 

20 throughput problems exist between host 14b and storage 

arrays 12b and 12c. Further, client 20a can determine the 
configuration of storage array 12e including the version of 
software code or update used by 
that storage system. 

25 Referring to Fig. 2, storage management server 16 

includes a poller 3 0 which has the primary function of 
retrieving the pertinent information about each of storage 
arrays 12a- 12e. Upon initiation of management server 16, 
poller 30 will poll for and "discover" all connected hosts, 

30 storage arrays, and storage connectivity devices (e.g., 

Fibre Channel, hubs, switches) . Poller 30 will then begin 
monitoring basic status of the arrays, including how the 
storage arrays are defined and grouped, as well the status 



of communication with agents 18a-18e. The frequency of 
polling can be selected by the client or, if not defined, be 
set to a default value (e.g., every minute.) The discovery 
process should also include retrieval of the storage array 
5 configuration information. 

Once retrieved, the configuration information is 
stored (in object format) in Object Store 32 and poller 30 
polls the appropriate agent for a configuration change date 
and time. Upon recognizing a configuration change, poller 

10 3 0 will retrieve a current or "fresh" copy of the 

configuration. Poller 3 0 will also notify the Object Server 
34 (here, a Java Object Server), which is responsible for 
notifying registered clients of the new configuration. Of 
course, whenever poller 30 requests a status update of 

15 configuration, it will poll for additional agents and 
storage arrays which may have been recently added. 

Object Store 32 maintains persistence on its stored 
objects. Persistence is maintained for current status and 
configuration information for each storage array connected 

20 to the network. In addition, statistical information may be 
stored for all or selected ones of the storage arrays. 

Object Server 34 uses a security component 3 6 to 
first validate user ID codes and passwords from a client, 
and will then retrieve from Object Store 32, the current 

25 status of all storage arrays that the user is authorized to 
view. Object Server 34 updates the client with any new data 
until the client "unregisters . " 

Storage management server 16 can optionally include 
an HTTP server (web server) 38, which is used to establish 

30 initial connection with clients 20a-20e. In essence, HTTP 
server 38 allows clients to establish connection using the 
address of the HTTP server rather than the proxy (e.g., 
Voyager ) addre s s . 



Referring to Fig. 3, an example of an operator 
display screen 40 of the graphical user interface is shown. 
Display screen 40 includes a Java applet window 42 contained 
within a web-browser window 44 (here, NetScape) . Within 
5 applet window 42 are several sub-windows, including a site 
sub-window 46 which lists the storage arrays accessible by 
the user, while a session sub-window 48 provides the status 
information of the current session. Sub-window 5 0 provides 
a graphical representation showing the grouping and 

10 hierarchy of the storage arrays, so that the user is able to 
quickly understand the connectivity relationship of the 
storage arrays of interest. 

Referring to Fig. 4, in another embodiment, two or 
more storage management servers 16a, 16b can be used to 

15 distribute the monitoring load of the storage arrays. Each 
management server includes the structure and provides the 
functionality shown in Fig. 2 as described above. In these 
applications, however, because multiple management servers 
are used, a name server 60 is provided to help manage the 

20 distribution of the management resources. In particular, 
name server 60 determines which of the Object Stores of 
storage management servers 16a, 16b includes the information 
relating to the operation status of the storage arrays of 
interest. Note that from the viewpoint of clients 20a-20e, 

25 the use of multiple management servers is invisible. 

Other embodiments are within the scope of the 
claims. For example, in the above description, the storage 
management server is used in conjunction with a Symmetrix 
enterprise disk storage system. It is important to 

3 0 appreciate that the storage management server is equally 

applicable in use with other storage area networks and other 
storage devices, including those that are not Symmetrix 
devices . 
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1 1. A network architecture comprising: 

2 a storage system including a plurality of storage 

3 devices; 

4 a plurality of host computers, each host computer 

5 transmitting data to and retrieving data from one or more of 

6 the plurality of storage devices; 

7 a plurality of clients; and 

8 a storage management server connected between the 

9 plurality of clients and the plurality of storage devices, 

10 the storage management server providing information relating 

11 to the operation status of the storage devices to at least 

12 on of the clients. 

1 2 . The network architecture of claim 1 wherein the 

2 storage management server includes : 

3 a poller for gathering the information relating to 

4 the operation status of the storage device; and 

5 a central repository for storing the information 

6 relating to the operation status of said one of the storage 

7 devices; and 

8 an object server for distributing the information 

9 relating to the operation status of the storage devices to 
10 the clients . 

1 3 . The network architecture of claim 1 wherein the 

2 poller polls each of the storage devices at predetermined 

3 intervals to maintain the current status of the operation of 

4 each of the storage devices. 

1 4 . The network architecture of claim 3 wherein the 

2 predetermined interval is less than or equal to one minute. 
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1 5. The network architecture of claim 1 wherein the 

2 storage management server further provides information 

3 relating to the operation status of storage connectivity 

4 devices which connect storage devices to the clients . 

1 6. The network architecture of claim 5 wherein the 

2 storage management server includes: 

3 a poller for gathering the information relating to 

4 the operation status of the storage device and storage 

5 connectivity devices; and 

6 a central repository for storing the information 

7 relating to the operation status of said one of the storage 

8 devices and storage connectivity devices; and 

9 an object server for distributing the information 

10 relating to the operation status of the storage devices and 

11 storage connectivity devices to the clients. 

1 7. The network architecture of claim 6 wherein the 

2 poller polls each of the storage connectivity devices at 

3 predetermined intervals to maintain the current status of 

4 the operation of each of the storage connectivity devices. 

1 8 . The network architecture of claim 1 wherein the 

2 storage management server further includes a security 

3 component for limiting access by a client to one or more of 

4 the storage devices. 

1 9. The network architecture of claim 1 wherein the 

2 storage management server further includes a web server for 

3 communicating with the plurality of clients. 



- 12 - 



1 10. The network architecture of claim 1 wherein 

2 each of the clients includes a graphical user interface for 

3 displaying the information relating to the operation status 

4 of the storage devices. 

1 11. The network architecture of claim 1 wherein the 

2 plurality of host computers are of different types. 

1 12 . The network architecture of claim 1 wherein the 

2 plurality of storage devices are of different types. 

1 13 . The network architecture of claim 1 further 

2 comprising a plurality of storage management servers 

3 connected between the host computers and the plurality of 

4 clients, each storage management server, providing 

5 information relating to the operation status of said one of 

6 the storage devices to at least one of the clients. 

1 14 . The network architecture of claim 1 wherein 

2 each of the storage management servers includes : 

3 a poller for gathering the information relating to 

4 the operation status of the storage device; and 

5 a central repository for storing the information 

6 relating to the operation status of said one of the storage 

7 devices; and 

8 an object server for distributing the information 

9 relating to the operation status of said one of the storage 
10 devices to at least one of the clients. 
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1 15. The network architecture of claim 14 further 

2 comprising a name server, connected to each of the plurality 

3 of storage management servers, to determine which of the 

4 central repositories of the plurality of storage management 

5 servers includes the information relating to the operation 

6 status of said one of the storage devices. 

1 16. A method of managing a storage system including 

2 a plurality of storage devices, the storage system 

3 communicating data to and from a plurality of host 

4 computers, the method comprising: 

5 providing a storage management server between a 

6 plurality of clients and the plurality of storage devices; 

7 collect, from the storage management server, 

8 information relating to the configuration of the storage 

9 system; and 

10 providing by the storage management server, the 

11 information to at least one of the clients. 

1 17. The method of claim 16 wherein providing 

2 information relating to the operation status of the storage 

3 devices includes using a poller to gather the information 

4 relating to the operation status of the storage device, the 

5 method further comprising storing the information relating 

6 to the operation status of said one of the storage devices 

7 in a central repository of the storage management server. 

1 18. The method of claim 16 wherein providing 

2 information relating to the operation status of the storage 

3 devices includes using an object server to distribute the 

4 information relating to the operation status of the storage 

5 devices to the clients. 
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1 19. The method of claim 16 wherein the poller polls 

2 each of the storage devices at predetermined intervals to 

3 maintain the current status of the operation of each of the 

4 storage devices. 

1 20. The method of claim 16 further comprising 

2 providing information relating to the operation status of 

3 storage connectivity devices which connect the hosts to the 

4 storage devices. 
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MULTIPLE STORAGE ARRAY CONTROL 
Abstract of the Disclosure 

A network architecture facilitates the monitoring 
and control of a number of storage systems (e.g., arrays) 1 
one or more clients. In particular, the network 
architecture includes at least one storage management 
server, connected between the storage systems and a number 
of clients, for providing operation status information 
associated with the storage systems as well as storage 
connectivity devices to at least one of the clients. 
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